test(e2e, playwright): added e2e test for /todo help command #238
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #238 +/- ##
======================================
Coverage 6.42% 6.42%
======================================
Files 11 11
Lines 1712 1712
======================================
Hits 110 110
Misses 1594 1594
Partials 8 8 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thanks for awesome addition @rahulsuresh-git. Tests LGTM, just some comments around formatting and naming
|
@toninis Any ideas why we need to press "Approve and run" on each commit of this forked repo PR? Workflows for this repo are here https://github.com/mattermost/mattermost-plugin-todo/tree/master/.github/workflows. Not sure which ones are the "2 workflows awaiting approval"
|
This is the organisation default behaviour on forked PRs for security reasons @mickmister |
|
@toninis Is this rule enforcement new? I know there is a "first-time contributor" check that has been enabled for a while, though I'm not used to seeing this for every commit |
mattermost/mattermost#25616 |
There was a problem hiding this comment.
Nice work on this @rahulsuresh-git 🎉 Looks really good. I have some comments for discussion on how we should organize the tests as a whole, and want to discuss the assertions we're making on the help command test
| postMessage('/todo help', page); | ||
|
|
||
| // # Grab the last post | ||
| const post = await getLastPost(page); | ||
| const postBody = post.locator('.post-message__text-container'); | ||
|
|
||
| // * Assert /todo add [message] command is visible | ||
| await expect(postBody).toContainText('add [message]'); | ||
|
|
||
| // * Assert /todo list command is visible | ||
| await expect(postBody).toContainText('list'); | ||
|
|
||
| // * Assert /todo list [listName] command is visible | ||
| await expect(postBody).toContainText('list [listName]'); | ||
|
|
||
| // * Assert /todo pop command is visible | ||
| await expect(postBody).toContainText('pop'); | ||
|
|
||
| // * Assert /todo send [user] [message] command is visible | ||
| await expect(postBody).toContainText('send [user] [message]'); | ||
|
|
||
| // * Assert /todo settings summary [on, off] command is visible | ||
| await expect(postBody).toContainText('settings summary [on, off]'); |
There was a problem hiding this comment.
@hanzei Curious what you think about this test. It could be a bit brittle since it's checking the output in this much detail. Essentially if anything changes about the help text (other than the order of commands in the output), this test will break.
We would also want to make this test fail if a new command is added to the help text, so maybe we want to count the number of lines somehow?
I think this is a difficult problem to solve, since we mainly want e2e tests to be "this feature generally works correctly", rather than "this feature's output looks exactly like this"
There was a problem hiding this comment.
Great work on this @rahulsuresh-git! I have a few comments more comments on the implementation. I think we can make the tests a bit less brittle by making their assertions less specific, and some suggestions on code organization
| const postBody = lastPost.locator('.post-message__text-container'); | ||
| return postBody; |
There was a problem hiding this comment.
Sometimes we'll want to parse other things from the post and not just the body. Maybe we can create a separate function getLastPostBody or getLastPostText and have it call getLastPost.
I'm mainly trying to be weary of any changes to the "meaning" of functions that are already in files being shared among the plugin projects. That's one of the main focuses on the e2e framework. This is essentially an externally facing API for other plugins to use.
|
Removing myself from the list of reviewers until the comments of @mickmister are resolved. |
|
Closing this PR as the changes are moved to this single PR: #254. |
Summary
This introduces a Playwright end-to-end (e2e) test to check if the
/todo helpcommand functions as expected. We verify the functionality of the various commands available for the Todo plugin.I've also made a few changes to improve clarity and readability.
Verified that the test passes locally.